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ABSTRACT 



This thesis presents a computer assisted, comparative analysis of 
empirical, maximum likelihood and exponential procedures for estimating 
reliability. The deviations of the estimators from the true reliability, 
when the underlying failure rate is monotone, are compared using the 
Kolmogorov-Smirnov family of statistics . The behavior of the distribu- 
tions of these deviations, for various sample sizes and failure rates is 
examined. Finally, the considerations of when to use which estimator 
are discussed. 
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I. INTRODUCTION 



Assumptions concerning the form of the underlying failure dis- 
tribution are made in most analyses of reliability problems. When 
these assumptions are in error the conclusions reached may be gross- 
ly in error. 

Some alternatives to making assumptions about the form of the 
underlying failure distribution are maximum likelihood estimation and * 
empirical estimation. 

This work presents a comparative analysis of empirical, maximum 
likelihood, and the often used exponential procedures when the under- 
lying failure distribution has a monotone non-decreasing failure rate. 

The maximum likelihood estimator is formed under the assumption that 
the underlying failure rate is monotone non-decreasing. The hypothesis 
that a sample of failure times comes from a distribution having a mono- 
tone failure rate can be tested using the method contained in Ref. 1. 

The relative quality of the three estimators was examined using 
the Kolmogorov-Smirnov family of statistics . 

The following notation and definitions are used in the succeeding 
sections. Let F be a right continuous distribution such that F (0“) = 0. 

If F has a density f then r(t)= is the failure rate, where 
R(t)= 1 - F(t) is the reliability or the survival probability. Thus 



R(t) = exp 



[< 



r(z)dz 



A distribution has an increasing failure rate 
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(IFR) if r(t) is monotone non-decreasing in t, and has a decreasing 
failure rate (DFR) if r(t) is monotone non-increasing in t. 
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II. THE ESTIMATORS 



A. THE EMPIRICAL ESTIMATOR 

The empirical estimator of reliability, Remp(t), based on a 
sample of n ordered observations (tj<t^< ...<0 from a failure dis- 
tribution F is; 

Remp(t) =1.0 t < 1 1 

Remp(t) = (n-i)/n t^< t<t^ + ^ 

Remp(t) =0 t > t 

n 

B. THE EXPONENTIAL ESTIMATOR 

The exponential estimator of reliability based on the above sample 
is Rexp(t) = exp [-kt] , t >0, where 1/k is the maximum likelihood 
estimate of the mean of the exponential distribution and is equal to the 
sample mean. 

C. THE MAXIMUM LIKELIHOOD ESTIMATOR 

The derivation of the maximum likelihood estimator is dependent 
upon an assumption that the underlying distribution has IFR or DFR. 

The IFR case is outlined here. Both the IFR and DFR cases are pre- 
sented in detail in Ref. 1. 

Let t. < t 0 ^ . . . <t be a sample of n ordered observations from 
1 L n 

F, IFR. Using the fact that 



6 



R(t) = exp 
expressed as 



j-J* r(z)dzj and r(t) = the log likelihood may be 

as 

n n n 

L=log(n!)+ ^ log f(t.) = ^ logr(t.) - I£ *r (z)dz + log(n ! ) 

i=l i=l i=l ”°° 

The maximization of L, subject to r(t) monotone non-decreasing, 
yields as an estimator for r(t) 

-1 



Ht) 



= min max v-u (n-u)(t 

v>i+l u^i L J L 



. . - t + . . . + (n-v+l)(t - t _ ) 

u+1 u v v-1 



i= 1,2, . . . , n-1 and r(t )=Qo . For the remaining values oft,?(t)is 0 
n 

for 0 <;t <t, ,c D for t >t , and constant (right continuous) between 
1 n 

observations. The corresponding estimator, R(t), is obtained from 



R(t) = exp 



[< 



r(z)dz 



Reference 1 shows that r(t) is a consistent 



estimator of r(t) . 
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III. PROCEDURES 



To evaluate the estimators of R(t) , computer simulation (Fortran IV r 

IBM-360, W. R. Church Computer Center , Naval Postgraduate School) 

was used to generate samples of failure times, compute the estimators 

and. the -statistics used in their evaluation. 

Two parent distributions were used, the Weibull and the Erlang. 

-bt a 

The Weibull distribution has reliability function R(t) = e with shape 

parameter a>0 and scale parameter b>0. The failure rate of the 

Weibull distribution is r(t) = abt a ^ . When a=l the distribution is the 

exponential. For values of a > 1 the Weibull distribution has IFR. 

The Erlang distribution was used to investigate whether the results 

were dependent upon the parent distribution used. The Erlang distribu- 

fV-1 b a e -bx 

tion has reliability function R(t)-J ; dx where a > 0, b > 0 

and a is an integer. The failure rate of the Erlang distribution is 
ta-l b a e'kt 

r(t) = . When a=l the distribution is the exponential. 

When a > 1 the Erlang distribution has IFR and the failure rate is 
bounded above by b. 

Consider a sample of n independent, identically distributed 
realizations of Weibull failure times. The true reliability of a system 
with this underlying distribution is R(t) = e“ bta . Let Remp(t) be the 
empirical estimator of R(t) as computed from this sample and let 



Demp = sup 
t 



Remp(t) - R(t) 



Then Demp is a one-sample, two-sided. 
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Kolmogorov-Smirnov statistic. Computing Demp for each of one 



hundred independent samples and ordering the results yields an empiri- 
cal distribution of the statistic Demp. 

Repeating this procedure for each estimator gives empirical distri- 



butions for the statistics Dexp= sup 

t 



Rexp(t) - R(t) 



Dmle = sup 



Rmle(t) - R(t) 



and 



Let Fdemp(x) be the empirical distri- 



t 

bution function of the statistic Demp. Similarly, let Fdexp(x) and 
Fdmle(x) be the empirical distribution functions of Dexp and Dmle 
respectively. 

For a given set of Weibull distribution parameters and a given 
sample size, define; 



D 1 = s s p 1 


[^Fdemp(x) - Fdmle(x)j 


D = sup 


^Fdmle(x) - Fdemp(x)J 


D 2 = S “ P 


j^Fdmle(x) - Fdexp(x)J 


D 2 = SPp 


[Vdexp(x) - Fdmle(x)J 


D 3 = spp 


^Fdemp(x) - Fdexp(x)J 


D- = sup 


^Fdexp(x) - Fdemp(x)^J 



These' statistics are two-sample, one-sided, Kolmogorov-Smirnov 
statistics and can be used to test the following types of hypotheses; 

Ho: Fdemp(x) = Fdexp(x) against the alternative HI: Fdemp(x) > Fdexp(x). 

To define the critical region for this test, note that 
P jjDg < D (°<) J =1 - cx , where D + (cx.) i s the 1- percentile of the 
two-sample, one-sided, Kolmogorov-Smirnov statistic distribution. 
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Reference 2 gives an approximation to the limiting distribution of D as 



lim P 
n l’ > °° 
n 2~ >c ° 





/ni+ n 2 


D < x 


1 J 



= i-o<i i- e 



-2x 



where n^ and n^ are 



the sample sizes of the two distributions tested. 

For a test of the hypothesis Ho: Fdemp(x) = Fdexp(x) against the 
alternative HI: Fdemp(x) ^ Fdexp(x), the two-sample, two-sided 

^3 ' D 3 



Kolmogorov-Smirnov test may be used. The statistic is D^=max 
Reference 3 defines the limiting distribution of D and Ref. 2 gives the 
following approximation 



lim 

n^->co 

n 2-*°° 



D <x 






l +n 2 



l n 2 



= 1- ot = l-2e 



-2x 



for values of x sufficiently large. 
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IV. RESULTS 



The failure rates examined vary from r(t) =1.0, the case when the 

2 

underlying distribution is exponential, to r(t) = 2.139t . In each case 

the mean time to failure is equal to one . 

The important difference in these failure rates is their relative 

rate of change with t. Throughout the remainder of this discussion 

the term high failure rate refers to a failure rate with a relatively high 

2 

rate of change with t. Thus r(t) = t is a higher failure rate than 
r(t) = t. 

Table I contains the statistics D+ and , as defined in 

Section III. The critical values of these statistics, at a level of 

significance of .01, are D + . (.01) = D~ (.01)= 0.215 . At a level 

cnt cnt 

of significance of .05 D + . (.05) = D . (.05) = 0.173. The statistic 
cnt cnt 

D can be determined from the table by using the fact that D = max D + , D" 

The critical values of D at levels of significance of .01 and .05 . are 

D . (.01) = 0.230 and D , f .05) = 0. 192. 
cnt cnt 

At a level of significance of .01 the hypothesis Ho: Fdmle(x) = 
Fdemp(x) is rejected in favor of HI: Fdemp(x) ^ Fdmle(x) when 
D^> 0.230. Ho is rejected in favor of HI: Fdemp(x) > Fdmle(x) when 



> 0.215. 



Reversing the sense of the inequality in HI and replacing Dj by 
Dj the test becomes, reject Ho in favor of HI: Fdmle(x)> Fdemp(x) when 
D~> 0.215. 
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As all samples of Dexp, Demp and Dmle are of the same size 
and theKolmogorov-Smirnov statistics are independent of the distribu- 
tions being tested/ these critical values of D + and D can be used 
for all cases presented here. 



TABLE I 



Observed values of Dt = sup 
1 x 



D-, = sup 

1 X 



Fdmle(x) - Fdemp(x) 



Fdemp(x) - Fdmle(x)J and 
for sample size n and failure rate r(t) 



r (t) 


o 
1 — 1 

II 

G 


n= 15 


n= 20 


n= 25 


3 

II 

GO 

o 


D t 


D I 


D t 


D I 




D I 


D t 


D I 


D t 


D I 


1.0 


0.57 


0 


0.73 


0 


0.79 


0 


0.96 


0 


0.98 


0 


1.287t ,S 


0.47 


0 


0.66 


0 


0.76 


0 


0.91 


0 


0.97 


0 


4t 

2t - 1 


0.43 


0 


0.70 


0 


0.85 


0 


0.92 


0 


0.97 


0 


1 . 571 1 


0.47 


0 


0.‘66 


0 


0.74 


0 


0.91 


0 


0.97 


0 


1 . 794 1 1 * 5 


0.47 


0 


0.65 


0 


0.74 


0 


0.91 


0 


0.97 


0 


2 . 139 t 2 


0.46 


0 


0.65 


0 


0.74 


0 


0.91 


0 


0.97 


0 



In all cases presented in Table I the value of is zero and the 
value of is significantly large which implies that the empirical 
estimator deviates less from the true reliability than does the maxi- 
mum likelihood estimator. 



As the sample size increases/ it is expected that the deviations 
of both estimators will decrease. The data in Table I indicate that 
the deviations of Remp(t) are decreasing faster than the deviations of 
Rmle(t) . 
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Before examining some of the sample points of the distributions 
Fdemp(x) and Fdmle(x), it should be noted that the sample 100 th 
percentile of Fdmle, denoted by Dmle(lOO), is the maximum devia- 
tion of Dmle for all samples of fixed size n with the same under- 
lying failure rate. Similarly, Dmle(l) is the minimum deviation and 
Dmle(lOO) - Dmle(l) is the sample range of Dmle. 

As Demp is a one-sample, two-sided Kolmogorov-Smirnov sta- 
tistic, it is independent of the distributions tested and will vary 

only with sample size. This is not true of Dexp and Dmle. 

2 

When the failure rate is 2.139t and the sample size increases 
from 10 to 30, Demp(lOO) decreases from .4120 to .2690, a reduction 
of about 35% in maximum deviation. Over the same range of sample 
sizes Dmle(lOO) decreases from .5426 to .4470, a reduction of about 
15% in the maximum deviation. For the same changes in sample size, 
Dmle(l) varies from .1816 to .1810 while Demp(l) varies from .1210 
to .0671. This implies that the entire range of Demp moves toward the 
origin as the sample size increases, while only the right hand tail 
of Dmle moves with changes in sample size. When the failure rate is 
constant the variation in Dmle(lOO) is from .6956 to .6165 while the 
variation in Dmle(l) is from . 1974 to .1899. 

The data in Table II indicate that the deviations of the expon- 
ential estimator are stochastically smaller than the deviations of the 
maximum likelihood estimator for all failure rates tested. 
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TABLE II 



Observed values of D* = sup jVdmle(x) - Fdexp(x)J and 
= sup ^Fdexptx) - Fdmle(x)j for sample size n and failure rate r(t) 



r (t) 


n = 10 


n = 15 


n = 20 


n = 25 


n = 30 




D 2 


■>; 


D 2 




D 2 




D 2 






1.0 


0 


0.88 


0 


0.94 


0 


0.93 


0 


0.99 


0 


0.99 


1.287 r 5 


0 


0.84 


0 


0.82 


0 


0.88 


0 


0.95 


0 


0.98 


4t 

2t- 1 


0 


0.83 


0 


0.92 


0 


0.93 


0 


0.95 


0 


0.98 


1. 571 t 


0 


0.63 


0 


0.64 


0 


0.78 


0 


0.82 


0 


0.93 


1.794 t 1 - 5 


0.03 


0.47 


0.08 


0.47 


0.05 


0.57 


0.02 


0.76 


0.01 


0.81 


2.139 t 2 


0.13 


0.30 


0.12 


0.31 


0.09 


0.37 


0.01 


0.51 


0.04 


0.64 



For a given failure rate, the magnitude of increases with 
sample size and for a fixed sample size it decreases as the failure 
rate increases. 

The behavior of the distributions of Dexp and Dmle are best 
illustrated graphically. Figures 1 through 4 depict a smoothed form 
of the empirical distributions as sample size and failure rate vary. 
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Fig. 1. Fdexp and Fdmle for n = 30 and r(t) =1.0 

Note: Dexp(l) = 0.0001 Dexp (100) = 0 . 1651 

Dmle (1)= 0.1899 Dmle (100) = 0 . 6165 




2 

Fig. 2. Fdexp and Fdmle for n = 30 and r(t) = 2 . 139t 
Note: Dexp (1) = 0.2610 Dexp (100) = 0.3565 

Dmle (1) = 0.1814 Dmle (100) = 0.4473 



15 



1.0 



Fdmle 




0 .2 .4 .6 



D 



9 

Fig. 3. Fdexp and Fdmle for n = 10 and r(t) = 2. 13 9t 
Note: Dexp (1) = 0 .2399 Dexp (100)= 0.4291 

Dmle (1)=0.1816 Dmle (100) = 0.5462 




Fig. 4. Fdexp and Fdmle for n = 10 and r(t) =1.0 
Note: Dexp (1) = 0 . 0010 Dexp (100) = 0 . 3571 

Dmle (1) = 0 . 1979 Dmle (100 )= 0 . 6456 
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As would be expected, the exponential estimator performs best 
when the failure rate is low and the sample size is large. As the 
failure rate increases Fdexp moves to the right and the range of Dexp 
decreases . 

The range of Dmle decreases as sample size increases or as 
failure rate increases. The first percentile of the distribution is rela- 
tively insensitive to change in either sample size or failure rate. 

For all cases, Ho: Fdmle(x) = Fdexp(x) can be rejected in favor 
of HI: Fdexp (x) > Fdmle(x) at level .01. 

The data in Table III indicate that the exponential estimator 
deviates less from the true reliability than does the empirical esti- 
mator when the failure rate is constant or nearly constant. When the 
failure rate is constant, D* = 0 and increases with the sample 
size, n. This is consistent with the theory as the maximum likeli- 
hood estimate of the parameter of the exponential distribution is the 

reciprocal of the sample mean and lim x= M . 

n-»oo 
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TABLE III 



+ 

Observed values of D = sup 
^ x 

D = sup 
J x 




j^Fdemp(x) - Fdexp(x)j 



and 



for sample size n and failure rater(t) 



r(t) 


n= 10 


n = 15 


n = 20 


n = 25 


n = 30 


»: 


D 3 




D 3 


D I 


D 3 




D 3 


■>; 


D 3 


1.0 


0 


0.72 


0 


0.74 


0 


0.76 


0 


0.80 


0 


0.78 


1 . 287 1* 5 


0 


0.60 


0 


0.43 


0.05 


0.29 


0.01 


0.23 


0.08 


0.25 


4t 

2t- 1 


0 


0.63 


0.02 


0.43 


0.02 


0.20 


0.06 


0.26 


0.13 


0.15 


1 . 57 1 t 


0.08 


0.24 


0.30 


0.03 


0.43 


0.06 


0.55 


0.01 


0.66 


0 


1 . 794 1 1 ‘ 5 


0.30 


0.03 


0.63 


0 


0.66 


0.01 


0.88 


0 


0.92 


0 


2.139 t 2 


0.52 


0 


0.82 


0 


0.83 


0 


0.95 


0 


0.98 


0 



As the failure rate becomes larger, D decreases to zero. D + 
increases with both sample size and failure rate. When the failure 
rate is not constant, decreases with sample size. 

Figures 5 through 8 show the behavior of Fdemp and Fdexp with 
changes in failure rate and sample size. 
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Fig. 5. Fdexp and Fdemp for n = 1*0 and r (t) =1.0 

Note: Dexp (1) = 0.001 Dexp (100) = 0 .3571 

Demp(l)= 0.1207 Demp (100)= 0.4122 




, . 2 

Fig. 6. Fdexp and Fdemp for n = 10 and r(t) = 2.139t 

Note: Dexp (1) = 0.2399 Dexp (100) = 0 .429 1 

Demp(l)= 0.1207 Demp (100) = 0.4122 
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Fig. 7. Fdexp and Fdemp for n = 30 and r (t) = 1.0 

Note: Dexp(l) = 0.0001 Dexp (100) = 0. 1651 

Demp (1)= 0.0670 Demp (100) = 0. 2690 
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Fig. 8. Fdexp and Fdemp for n = 30 and r(t) = 2 . 139 t 

Note: Dexp (1) = 0.2610 Dexp (100) = 0.3565 

Demp(l) = 0.0670 Demp (100) = 0.2690 

Note that the distribution Fdemp is invariant with changes in 
failure rate and varies only with the sample size. The changes in 
D* and D^ for a given sample size occur because the distribution 
Fdexp varies with r(t). As r(t) increases Fdexp moves to the right 
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for all sample sizes examined. For high failure rates the left end 



point of Fdexp moves away from the origin as sample size increases 
while the right end point moves toward the origin. 

The stochastic ordering of Fdexp and Fdemp varies with the 
underlying failure rate with Fdemp > Fdexp for higher failure rates. 
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V. CONCLUSIONS 



The statistics in Tables I and II support the hypothesis that the 
empirical and the exponential estimators give better estimates of 
reliability than does the maximum likelihood estimator for the cases 
considered . 

Comparing the exponential and the empirical estimators, it can 
be seen that while both sample size and failure rate affect the mag- 
nitudes of D + and D , the failure rate of the underlying distribution 
is the factor that determines the stochastic ordering of the distributions 
Fdemp and Fdexp. 

The exponential estimator would be the logical choice when the 
underlying failure rate is constant, or nearly constant, while the 
empirical estimator performs better when the failure rate is higher. 

In considering the question of when to use which estimator it 
is necessary to investigate the nature of the underlying failure rate. 

Reference 4 contains several tests for the validity of the assump- 
tion that the underlying life distribution is exponential. 

If physical considerations or tests, such as those presented in 
Ref. 4 substantiate the assumptions that the underlying distribution 
is exponential, then the exponential estimator of reliability should be 
used. If the exponential assumption cannot be substantiated, the 
empirical estimator should give better estimates of the system reli- 
ability than the exponential estimator. 
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Physical considerations of some systems indicate that the failure 
rate is not monotone but conforms to the bathtub model where the 
failure rate is initially high but decreasing, relatively constant for 
the middle age life periods, and increasing with old age. Some 
electronic components are believed to behave in this manner, experi- 
encing a "bum-in" period at the onset of their working life cycle. 

The empirical estimator should be particularly useful in cases 
where the failure rate is not monotone as no assumptions about the 
form of the underlying distribution, or its failure rate are required. 
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